System and method for preview and selection of words

ABSTRACT

A system, method and computer-readable medium that previews words and commands recognized by a continuous stroke recognizer based on input into a virtual keyboard layout is presented. The system and method allows a user to see the word that will be recognized or output before the user commits the continuous stroke input to the system. Further, after recognition and output, the system presents the recognized output word and the closest word candidates in a correction display integrated with the virtual keyboard layout. Among other things, the user is provided with options to change the current output word by selecting another word candidate on the correction display, and to delete the current output word by selecting the representation of the current output word on the correction display.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application a continuation of U.S. patent application Ser. No.13/011,064, filed on Jan. 21, 2011, which is a continuation of U.S.patent application Ser. No. 11/796,654 filed on Apr. 27, 2007, now U.S.Pat. No. 7,895,518 and entitled “System and Method for Preview andSelection of Words,” which applications are incorporated herein byreference in their entireties.

TECHNICAL FIELD

The present invention relates generally to the preview of text entry forcomputers, PDA's, cellular phones, smartphones, tablet PC's and otherdevices, particularly text entry via a continuous stroke recognizerbased on a virtual keyboard layout.

BACKGROUND

The increasingly limiting bottleneck to information flow in ultra-mobilecomputing results from, among other things, the lack of efficient userinterfaces on these devices. In comparison to desktop or laptopcomputers, today's handsets and tablet PCs are far more cumbersome touse. One solution is “shape writing” where a user inputs shorthandgestures that are defined by a virtual keyboard layout. Such shorthandgestures are continuous strokes that are recognized by a continuousstroke recognizer based on a virtual keyboard. Examples of such systemsare described in Kristensson, P. O., Design and Evaluation of aShorthand-Aided Soft Keyboard, Master's thesis, Linkoping University(2002); U.S. Pat. No. 7,251,367, System and method for recognizing wordpatterns based on a virtual keyboard layout (Zhai); U.S. Pat. No.7,098,896, System and method for continuous stroke word-based text input(Kushler et al.); and Kristensson, P. O. and Zhai, S., SHARK.sup.2: ALarge Vocabulary Shorthand Writing System for Pen-based Computers, Proc.17.sup.th Annual ACM Symposium on User Interface Software and Technology2004, 43-52 (2004). Such systems and methods do not provide previewsbased upon uncompleted strokes; do not provide preview or correctiondisplays integrated in a virtual keyboard (for example, in Kushler etal., a display is incorporated in a word processing application, and inKristensson, P. O. and Zhai, S., a user must utilize a pull-down menu);and do not provide for deletion of words in such a preview or correctiondisplay.

Similarly, U.S. Pat. No. 7,706,616, System and method for recognizingword patterns in a very large vocabulary based on a virtual keyboardlayout (Kristensson et al.) and U.S. Pat. No. 7,382,358, System andMethod for Continuous Stroke Word-Based Text Input (Kushler et al.),present systems or methods for recognizing continuous strokes based on avirtual keyboard layout, but do not provide a preview of the recognizedcommand for an uncompleted stroke; an integrated correction display; oran option to delete or cancel an output word by selecting arepresentation of the current word in a selection display. United StatesPatent Application 20050114115, Typing accuracy relaxation system andmethod in stylus and other keyboards (Karadis et al.), discloses asystem and method to analyze tapping patterns and automatically correctmis-tapped words, but does not use a continuous stroke recognizer. Arvo,J. and Novins, K., Fluid sketches: continuous recognition and morphingof simple hand-drawn shapes. Proceedings of the 17.sup.th Annul. ACMSymposium on User Interface Software and Technology 2000: 73-80 (2000),presents a software system to recognize pen-gestures while a user isarticulating the pen-gesture, but works with a limited set ofpen-gestures. See, also, Zhai, S. and Kristensson, P. O., ShorthandWriting on Stylus Keyboard. Proceedings of the ACM Conference on HumanFactors in Computing Systems 2003, 17-24 (2003).

Briefly, shorthand gesturing on a virtual keyboard works by having auser gesturing a pen or finger trace on a virtual keyboard layout (suchas the virtual keyboard layout illustrated in FIG. 3 of U.S. Pat. No.7,251,367 (Zhai), which patent is incorporated herein by reference).When the user lifts up the pen or finger, the system recognizes the penor finger trace. The pen or finger trace is then compared with thegeometric trace or pattern connecting the letter keys of a template wordin a lexicon. This comparison is repeated for all words in a lexicon.The word that most closely matches the user's pen or finger trace isreturned as the recognized word.

Some users may not feel confident when shorthand gesturing because theinput interface is new to them. Such users, including novice users,simultaneously need to learn a new shorthand gesture text inputinterface and verify that they are on the right track.

Another problem with existing shorthand gesturing is that no recognitionor feedback from the system is communicated to the user while the useris producing a shorthand gesture to indicate the user is or is not onthe right track.

Existing shorthand gesturing does not provide for repeated and/orcontemporaneous recognition of a user's pen or finger trace. Inaddition, there is a need to avoid situations in which a recognized wordis sent to the associated application (such as a word processingapplication or e-mail client) without confirmation that it is the wordintended by the user, and a need to provide a user with the opportunityto cancel the process if the user has made a mistake.

Users may also want to send computer system commands such as “Copy,”“Print” and “Track Changes” with a shorthand gesture. In this regard,reference is made to U.S. Pat. No. 7,487,461, System and method forissuing commands based on pen motions on a graphical keyboard (Zhai etal.), which does not provide a preview of an uncompleted stroke, or adisplay integrated with a virtual keyboard, or an option to delete anunintended word.

When a user is gesturing a shorthand gesture, the need for a preview isalso desired. In addition, computer system commands perform operationsthat affect the user's application and can be destructive to the user'swork. For example, the computer system command “Close” may close theuser's application and result in data loss for the user.

Clearly there is a need for both novice and expert users of shorthandgesturing for a preview of the word or system command input by a user asa pen or finger trace, and for other solutions as set forth herein.

SUMMARY

The present invention satisfies such needs, and presents a system, acomputer program product comprising a computer-readable medium havingcomputer-executable instructions, and an associated method forindicating to users the currently recognized word and what word will beoutput to the computer application. Prior continuous stroke recognizersrecognize and output a recognized word after a user has completed astroke. In one aspect of the present invention, a user's continuousstroke is constantly recognized while a user is inputting a continuousstroke, and a preview (e.g. a continuously updated preview) of thecurrent matching words of users' continuous strokes is displayed on adisplay integrated in a virtual keyboard, in one aspect through asoftware program or device. In another aspect, after the user hascompleted a stroke, the system and method outputs the recognized outputword, and presents the recognized output word and the closest wordcandidates in a correction display integrated with the virtual keyboardlayout. In yet another aspect, the user may change the current outputword by selecting another word candidate on the correction display. Theuser may then delete the current output word by selecting therepresentation of the current output word on the correction display.

A user can input a stroke by for example pushing and sliding a pen orfinger on a touch-screen or electronic tablet surface, and a user cancomplete an input stroke by, for example, lifting a pen (or stylus) orfinger from a touch-screen or electronic tablet surface.

The present system repeatedly recognizes the user's continuous strokeinput before the stylus or finger is lifted. If a set of matching wordsis found, the best matching words are displayed to the user in acorrection display. In one aspect of the invention, the currentlyrecognized preview word has a distinct background color.

In another aspect of the invention, once a user completes a stroke, forexample by lifting up the pen or finger from a touch-screen orelectronic tablet surface, the best matching word is recognized by thecontinuous stroke recognizer and shown in a correction display with orwithout the next-best matching words. In another aspect, the user maythen delete the current output word by selecting the representation ofthe output word in the correction display. The user may change thecurrent output word by selecting a representation of one of the othernext-best matching words in the correction display.

In one aspect of the invention, the correction display is always presentand is or can be integrated with a virtual keyboard. Integrating thecorrection display with a virtual keyboard has the advantage that thepreview word is in close vicinity to the virtual keyboard which is inthe user's focus of attention.

The correction display, in one aspect, can show more than just onepreview word. Since many words may be likely to match a user's input(completed or uncompleted input), it is beneficial to show someadditional closely matched word candidates and not only the closestword. In one aspect of the invention, after the user has lifted up thepen or finger from the virtual keyboard, the preview word is sent to theuser's operating system or application. The latest word that was sent tothe application is the current output word. If the user later realizesthat the wrong word was recognized, the user is provided with anopportunity to change the current output word into another similar orclose word candidate shown on the correction display by selecting it.

In one aspect of the invention, if the correction display does not showthe word that the user intended to write after lifting the stylus offinger, the user may decide to start over. It is cumbersome to delete aword that is already sent to the application. To make this easier, oneaspect of the invention provides that the user can click on therepresentation of the current output word in the correction display todelete the current output word.

In another aspect of the invention, when a user is producing a shorthandgesture and notices the intended word shown as a preview in thecorrection display, the user can release the pen or finger even if theshorthand gesture is not completed. The lexicon of recognizable wordsmay contain substrings of words in the language. For instance, thelexicon can contain the substring “th” which is a prefix for the word“the”. If the user is producing a shorthand gesture for “the,” thesystem in one aspect will notice when the user's pen or finger trace isrecognized as the word pattern for the prefix “th” that the user islikely to intend to write the word “the,” since the prefix “th” ismapped to the word “the” in the system's lexicon. If the user noticesthe preview, the user can release the pen or finger for some shorthandgestures of the intended word. This is also an advantage to expert andrepeat users who can learn the minimal shorthand gesture necessary forthe system to recognize the intended word.

When a user is producing a shorthand gesture, the user may notice thatan alternative word other than the most likely preview word is shown inthe correction display. For example, if the user is intending to writethe word “the,” the word “three” may show up as the primary preview wordand the intended word “the” as the secondary preview word. The usercannot release the pen or finger at that point since that means that theunintended word “three” would be output. Instead, in one aspect of theinvention, the user can quickly move the pen or finger towards arepresentation of the intended word shown as a preview in the correctiondisplay. This is possible because the method and system in one aspectwill recognize or notice that the user moved outside the virtualkeyboard at or above a predetermined threshold speed such that the usermost likely intended to directly select a preview word in the correctiondisplay.

When a user is producing a shorthand gesture, the user may accidentallyproduce a wrong trace. In such a case, the user will not see a word inthe preview display that matches the intended word. The user may at thistime want to cancel the shorthand gesture before it is recognized as aword and sent to an application. In such a case, one aspect of theinvention provides that the user may move the pen or finger to left,right or bottom outer edges of the virtual keyboard. Another alternativeis a special cancel icon or cancel area where the user can drag tosignal cancel.

In one aspect, the user can also delete the current output word byselecting a representation of it in the correction display.

A constantly updated display may take visual attention away from theuser. To prevent this adverse effect, in one aspect the preview is onlydisplayed if the user inputs the continuous stroke slowly. In anotheraspect, any movement slower than certain letter keys per second triggerspattern recognition and visual preview. In one example embodiment, thischeck is performed by the system every 20 ms (50 Hz).

In one embodiment, an uncompleted stroke can be cancelled by selecting arepresentation of cancel input. For example, a user can slide a penoutside the left, right or bottom virtual keyboard edges to signalcancellation.

It is possible to enter both words and commands in one shape writingsystem. For example, see, U.S. Pat. No. 7,487,461, System and method forissuing commands based on pen motions on a graphical keyboard (Zhai etal.).

In one exemplary embodiment, suppose a user wants to issue the command“Copy”. The user may start by landing the pen or stylus on the “Command”key or any other trigger key, and then drag it to the first letter keyin the command, in this case the “C” key. Since Command-C matchesanother command (“Cut”) that is shorter and/or more frequently used, theinvention then previews “Cut” (FIG. 12A). Other commands that also matchthe sequence “Command-C”, in this case “Copy”, “Close” and “Comment”,are shown in a list of alternatives to the left in the correctiondisplay. To enter “Copy” the user may either quickly slide the pentowards “Copy” in the leftmost, or continue to gesture towards thesecond letter key “O”. Since Command-C-O matches “Copy” the command“Copy” is now previewed (FIG. 12B). Then, when the user lifts up thepen, the “Copy” command is issued.

Any commands shown in the alternative list to the left of the centerpanel can be directly selected by quickly dragging the pen towards thecommand name. The method and system of the present invention in oneembodiment can unambiguously separate gestures from alternative listselections since once the pen tip leaves the keyboard area it does notconstitute a valid gesture. Since movement dynamics are taken intoaccount when deciding if pattern recognition and update of the previewshould be performed, the alternative list will not suddenly changedespite the user gesturing over the keyboard while heading towards thedesired command.

In one aspect, the present invention comprises software which may beloaded and run on a computer, PDA, cellular phone, smartphone, tablet PCand other electronic devices. In another aspect, the invention comprisesa computer, PDA, cellular phone, smartphone, tablet PC and otherelectronic or mobile device which provides some or all of thefunctionality disclosed herein.

The invention in one aspect provides a method of displaying a wordpreview in a continuous stroke system, comprising: receiving a firstpart or portion of a continuous stroke which has been input into avirtual keyboard; comparing the first part of the continuous stroke witha plurality of known word patterns which correspond to a plurality ofknown words; generating a preview of known words which correspond tothat first part of said continuous stroke; and displaying the preview ona display before the user has completed entry of the continuous stroke(e.g. before the user has lifted up his or her pen or stylus). Inanother aspect, the method comprises: receiving a second part or portionof said continuous stroke; comparing the first and second parts of thecontinuous stroke with such known word patterns corresponding to knownwords; displaying a preview of known words corresponding to the firstand said second parts; and displaying a preview on the display prior tocompletion of said continuous stroke. The method in one aspect providesfor an ongoing, continuously updated preview which is updated as a usermoves a pen, stylus or finger from location to location on a virtualkeyboard.

In various aspects of the invention, the preview may be displayed on adisplay integrated with the virtual keyboard (e.g. inside a softwarecomponent keyboard, adjacent or in close proximity to it, such asdirectly above), or positioned outside the virtual keyboard.

Another aspect of the invention provides a method comprising: receivinga completed continuous stroke; comparing the completed continuous strokewith a plurality of known word patterns which correspond to a pluralityof known words; displaying a plurality of known words which correspondto said completed continuous stroke; and displaying the known words onthe display.

In another aspect, the known words in the preview are alternativelyselectable by a user, and the method accepts a selection of a desiredknown word in the preview when a part of a continuous stroke intersectsthe desired known word in the preview (e.g. a pen, stylus or fingertouches the desired known word).

In yet another aspect, the continuous stroke system is associated with acomputer operating system (on a stationary computer, embedded device orserver, for example), and the desired known word is output to theassociated operating system. This may occur when a user sees a previewof a word and lifts up his or her pen, or when a user quickly (e.g. morequickly than expected) moves his or her stylus to a representation of apreview word (most likely an alternate next best matching word). If therate of movement is as expected, the method in one aspect will treatsuch movement as a continuation of the continuous stroke of the user.

In other aspects of the invention, a continuous stroke system isassociated with an active application (for example, a word processor, ane-mail client, instant messaging client, forms application or otherapplication running or capable of running on a computer, PDA, cellularphone, smartphone, tablet PC or other device), and the desired knownword is output to the associated active application.

Another aspect of the invention provides a method comprising: receivinginformation about movement dynamics (i.e. spatiotemporal information)concerning the continuous stroke; comparing this spatiotemporalinformation to expected values; and displaying a preview on the displayif this spatiotemporal information differs from expected values.Alternatively, the invention may accept the selection of a desired knownword in the preview if such spatiotemporal information differs fromexpected values.

In another aspect of the invention, the method comprises: receivingspeed information concerning the rate of movement of the continuousstroke; comparing the speed information to a predetermined value;outputting to an associated active application a known word in thepreview if the speed information is greater than the predeterminedvalue; whereby a known word in the preview may be selected by a user.

For example, in one embodiment, a correction display will update onlywhen a user moves a pen or finger slowly; if the pen or finger movesquickly (i.e. greater than a predetermined speed), the display remainsstatic. In this way, a user can select a preview word in the correctiondisplay by moving his stylus or finger quickly, for example. This avoidsa situation where a preview word is selected due to the fact that a usermoves at a normal rate toward a preview word in the current correctiondisplay without an intent to select such word, but instead with theintent to continue toward completion of the continuous stroke.

In other aspects of the invention, the method comprises terminating thereceiving of a continuous stroke if part of the continuous strokeintersects with an edge of the virtual keyboard; or terminating thereceiving of the continuous stroke if part of the continuous strokeintersects with an icon in or near the virtual keyboard (e.g. an icondesignated “cancel”).

Another aspect of the invention provides a method of defining wordpatterns of a plurality of known words by a plurality of input paths,wherein each path is associated with a word input into a virtualkeyboard; and matching the path of one or more parts of the continuousstroke with a corresponding word pattern.

In other aspects of the invention, the “words” described herein comprisecommands to an electronic device associated with the continuous strokesystem (e.g. “Cut”, “Copy”, “Close”). In one such aspect, the methodcomprises: comparing one or more parts of the continuous stroke with aplurality of known command patterns which correspond to a plurality ofknown commands; matching these part(s) of the continuous stroke to acorresponding command pattern; and executing an instructioncorresponding to said corresponding command pattern.

In various aspects of the present invention, the method, system andcomputer-readable medium comprises a computer, PDA, cellular phone,smartphone, tablet PC or mobile communications device having a virtualkeyboard; or in another aspect, a projection device for projecting thevirtual keyboard on a wall, screen or display; whereby a user may inputa continuous stroke through the projected representation of the virtualkeyboard.

The invention in one aspect provides a method of displaying a list ofcandidate words in a continuous stroke system having a virtual keyboardwith an integrated correction display, comprising: receiving acontinuous stroke which has been input into a virtual keyboard;comparing the continuous stroke with a plurality of known word patternswhich correspond to a plurality of known words; generating a pluralityof known words which correspond to the continuous stroke; and displayinga plurality of known words on a correction display integrated with thevirtual keyboard. In one aspect, a best matching word may be output toan associated active application. In another, the best matching word maybe deleted from (a) the associated active application or (b) thecorrection display if the best matching known word is selected by a useron the correction display.

The correction display may be adjacent to the keys of the virtualkeyboard. Words may be deleted from wherever they are output; forexample, if the word “the” is output to a word processor, MicrosoftWord, for example, and a user clicks on “the” on the correction display,the word “the” is deleted from the Word application.

In other aspects, a best matching word is displayed with a distinctgraphical attribute, such as an alternate color (i.e. different from theother preview words), or displayed with an associated icon. Theseaspects, as with all others, may include an associated computer, PDA,cellular phone, smartphone, tablet PC or mobile communications devicehaving a virtual keyboard; or a projection device for projecting thevirtual keyboard on a wall, screen or display.

Any of the foregoing aspects of the invention may be implemented througha computer-readable medium having computer-executable instructions forperforming one or more aspects of the invention. In addition, theinvention includes a system for performing one or more, or all, of theforegoing methods.

The general object of the invention is to provide a preview of words,phrases and/or commands based upon text entry at any given point intime. One advantage and improvement over current systems and interfaceswhich is provided by the present invention is repeated and/orcontemporaneous recognition of a user's pen or finger trace. If the penor finger trace resembles a shorthand gesture for a word, the improvedsystem and method shows the word as a preview on a correction displayintegrated with the keyboard.

By using this improved system and method, users gain an advantage byhaving knowledge that when they lift up their finger or pen, the wordthat is shown in the preview on the correction display is the word thatis recognized and will be sent to the user's application, for example,an email client.

The preview provides the further advantage of increasing a users'confidence and making the user more certain that the application isworking as intended. Further, support costs may be reduced since it iseasier for the user to guess what they should or may do.

For novice users, previews of computer system commands provide the userwith security and a sense of trust in the system.

For expert users such previews provide the user with confidence that theintended command will be sent to the application, even if the expertuser knows they did not produce the shorthand gesture accurately andthere is a high risk the intended shorthand gesture would normally notbe recognized by the continuous stroke recognizer.

These and other objects and advantages will become apparent from theforegoing and ongoing written specification, the accompanying drawingsand the appended claims.

BRIEF DESCRIPTION OF THE DRAWINGS

The various features of the present invention and the manner ofattaining them will be described in greater detail with reference to thefollowing description, claims, and drawings, wherein reference numeralsare reused, where appropriate, to indicate a correspondence between thereferenced items, and wherein:

FIG. 1 is a schematic illustration of an exemplary operating environmentin which the preview and selection system of the present invention canbe used.

FIG. 2 is a block diagram of a high-level hierarchy of the preview andselection system illustrated in of FIG. 1.

FIG. 3 is a flow chart that illustrates a decision step for determiningwhether a continuous stroke exceeds a predetermined speed.

FIG. 4 is a flowchart illustrating a method of determining whether auser desires to input a current preview word or, alternatively, tocontinue the continuous stroke input.

FIG. 5 is a flowchart illustrating a method of determining whether todisplay a preview of a recognized word or, alternatively, to clear ordelete a preview of a word.

FIG. 6 is an exemplary virtual ATOMIK keyboard illustrating the strokeinput of the word “these.”

FIG. 7 is an exemplary virtual ATOMIK keyboard illustrating a correctiondisplay.

FIG. 8 is an exemplary virtual ATOMIK keyboard illustrating a correctiondisplay displaying a preview of matching words in response touncompleted stroke input of the word “these.”

FIG. 9 is an exemplary virtual QWERTY keyboard illustrating a correctiondisplay displaying a preview of matching words in response touncompleted stroke input of the word “these.”

FIG. 10 shows an exemplary virtual QWERTY keyboard illustrating previewwords corresponding to only a substring of a word. In FIG. 10, theshorthand gesture for “th” has been input, and the preview of the mostlikely word “the” is illustrated in a correction display integrated withthe keyboard.

FIG. 11 shows an exemplary virtual QWERTY keyboard illustratingcancellation of the current input. The preview word “the” is cancelledin this illustration which shows movement of the pen trace towards theleft edge of the virtual keyboard layout.

FIGS. 12A and 12B illustrate an alternative design of a correctiondisplay integrated into a virtual keyboard, with a cancel inputrepresentation at the top left in the form of an icon.

DETAILED DESCRIPTION

The following definitions and explanations provide backgroundinformation pertaining to the technical field of the present invention,and are intended to facilitate the understanding of the presentinvention without limiting its scope:

ATOMIK”: Alphabetically Tuned and Optimized Mobile Interface Keyboard. Akeyboard layout that is optimized by an algorithm in which the keyboardwas treated as a “molecule” and each key as an “atom”. The atomicinteractions among the keys drive the movement efficiency toward themaximum. Movement efficiency is defined by the summation of all movementtimes between every pair of keys weighted by the statistical frequencyof the corresponding pair of letters. ATOMIK is also alphabeticallytuned, causing a general tendency that letters from A to Z run from theupper left corner to the lower right corner of the keyboard, helpingusers find keys that are not yet memorized. ATOMIK is one exemplaryvirtual keyboard that can be used in association with the presentinvention.

PDA”: Personal Digital Assistant. A pocket-sized personal computer. PDAstypically store phone numbers, appointments, and to-do lists. Some PDAshave a small keyboard, others have only a special pen that is used forinput and output on a virtual keyboard.

Virtual Keyboard”: A computer simulated keyboard with touch-screeninteractive capability that can be used to replace or supplement akeyboard using keyed entry. In a typical virtual keyboard keys may betapped serially with a stylus. It is also referred to as graphicalkeyboard, on-screen keyboard, or stylus keyboard.

Now turning to the drawings, FIG. 1 portrays exemplary environments inwhich the preview and selection system 10 can be installed. The computeror device in which the preview and selection system 10 is installed canbe a mobile device such as a PDA 15, cellular phone 20 or smartphone. Inaddition, the system and method 10 may be utilized with devices such asa tablet computer 25, touch screen monitor 30, electronic white board35, and digital pen 40.

The system and method may be utilized with device(s) using a continuousstroke recognizer based on a virtual keyboard layout or similarinterface for entry, represented by auxiliary device. Alternatively, inanother embodiment, the preview and selection system 10 can be saved ona suitable storage medium such as a diskette, a CD, a hard drive, orlike devices.

A high-level hierarchy of one embodiment of the word pattern recognitionsystem and method is illustrated in FIG. 2. The system and methodcomprises a continuous stroke recognizer 205 to capture a continuousstroke of a user on a virtual keyboard interface 200. An example ofcontinuous stroke input on a virtual ATOMIK keyboard interface 600 isshown in FIG. 6, which also illustrates a continuous stroke 605. In theembodiment illustrated in FIG. 2, the continuous stroke recognizer 205sends a preview set of words to a preview unit 210. The preview unit 210then displays the preview set of words in correction display 215. Thepreview set of words can be empty in which case no word is displayed inthe correction display 215. The preview set of words can consist of onlyone word in which case only one preview word is displayed in thecorrection display, and so on. The order of the steps in the embodimentsillustrated in the various drawing figures, including FIGS. 2, 3, 4 and5, may be varied depending upon the application and desired result oroutput.

In an alternative embodiment, a user is provided with the option ofdirectly selecting a preview word by extending the continuous stroke tointersect the representation of the intended preview word in thecorrection display. In an alternative embodiment, a representation of acancellation action is displayed in the correction display. In thatembodiment, the user cancels the uncompleted stroke input by extendingthe continuous stroke to intersect the representation of thecancellation action in the correction display.

When an input stroke is completed, the continuous stroke recognizer 205in FIG. 2 sends a final set of words to the correction display 215. Thecorrection display sends the best matching word to the output unit 220.The best matching word is then the current output word. FIG. 7 shows anexample of a correction display 615 associated with a virtual ATOMIKkeyboard 610 after the user has completed a stroke. In FIG. 7, thecurrent output word is “these”, and the words “those” and “therese” arenext best matching words. In this embodiment, the user is provided withan option to delete the current output word by selecting therepresentation of the current output word in the correction display.

The user may then change the current output word to another word byselecting it from the final set of words displayed in the correctiondisplay 215. If the user changes the current output word, a series ofedit actions are sent to the output unit. In one embodiment, the outputunit 220 will then synthesize a sequence of BACKSPACE key press eventsand inject them into the keyboard queue in the operating system. Forexample, if the current output word was “the”, three BACKSPACE key pressevents are synthesized and injected into the keyboard queue in order todelete the word “the” at the text caret position in the operatingsystem. Afterwards, the user's new current output word is sent to theoutput unit 220.

The process flowchart in FIG. 3 illustrates the preferred decisionprocess for uncompleted strokes in the preview system. Decision step 305determines if the uncompleted input stroke 300 is fast. In oneembodiment, this relative speed is determined by calculating the numberof spatial units the continuous stroke has moved per time interval. Ifthis number is more than a predetermined threshold, the stroke is fast.

Decision step 305 can be invoked in a number of ways. In one embodiment,decision step 305 is invoked whenever a continuous stroke is input.

In another embodiment, to ease the computational load on the system,decision step 305 is invoked at a fixed interval whenever a continuousstroke is inputted. For example, decision step 305 can be invoked every20 ms (50 Hz).

If the stroke is determined to be fast in decision step 305, the methodproceeds to step 400 (see FIG. 4) and decision step 405 in FIG. 4decides if a representation of a preview word is selected in thecorrection display 215. In one embodiment, decision step 405 is decidedby calculating whether the spatial coordinates of the input strokeintersects the graphical representation of a preview word in thecorrection display 215.

If decision step 405 decides a preview word is selected in thecorrection display 215, the current output word is changed in thisembodiment into a new preview word 415. For instance, if the currentoutput word is “these” and the continuous stroke is at decision step 305decided to be fast, and selected the word “those” at decision step 405,a message is sent to output unit 220 to delete the word “these” andreplace it with the word “those”. The correction panel is changed inthis example to reflect the change of current word. For example, thebackground color of the words in the correction display may be changedto reflect the current output word. It is not necessary that there be acurrent output word in all circumstances. For example, in oneembodiment, when a user gestures the shorthand for “the”, the system andmethod only shows a preview and does not output a word to an operatingsystem or application (MS Word, for example). Only after the user haslifted the pen, the word “the” is output to such OS or application. Inone example, if the user would choose alternative word “thee”, forexample, the system would not communicate to the output unit as no wordhas yet been sent to the output unit. In another example, however, acurrent output word may be sent to an associated application. Therefore,when a user is gesturing, the application may be constantly changingdepending on which word is currently previewed.

If the stroke is determined to be fast in decision step 305, the methodproceeds to step 400 (see FIG. 4) and decision step 410 in FIG. 4determines if a cancel input representation is selected in thecorrection display 215. In one embodiment, decision step 405 isdetermined by calculating whether the spatial coordinates of the inputstroke intersects the graphical representation of the cancel inputrepresentation in the correction display 215.

If decision step 410 determines that a cancel input representation isselected in correction display 215 by the uncompleted stroke 300, theuncompleted stroke 300 is cancelled 420, a STOP message is sent tocontinuous stroke recognizer 205, and the correction display 215 iscleared and shows no current word or any next-best matching wordcandidates. In one embodiment, a cancel input representation is a strokegoing outside the left, right or bottom virtual keyboard boundaries. Asan example, FIG. 11 illustrates an uncompleted continuous stroke 656that continues outside the virtual keyboard's 650 left edge. In FIG. 11,display 655 is illustrated adjacent to the top edge of the virtualkeyboard 650.

If at step 305 it is determined that the uncompleted stroke 300 is notfast, or if decision step 410 decides that the uncompleted stroke 300does not select the cancel input representation in correction display215, the method proceeds to step 500 (see FIG. 5) and decision step 505determines whether the input stroke matches at least one word in thecontinuous stroke recognizer 205. If the uncompleted stroke 300 does notmatch at least one word in the continuous stroke recognizer 205,correction display 215 is cleared at step 510 in FIG. 5. If theuncompleted stroke 300 does match at least one word in the continuousstroke recognizer 205, correction display 215 is cleared, and updated todisplay the current preview set of words 515 from the continuous strokerecognizer 205.

FIG. 8 illustrates an uncompleted stroke 626 on a virtual keyboard 620with an integrated correction display 625. The uncompleted stroke isrecognized by continuous stroke recognizer 205 to a ranked preview setof words: {“these”, “those”, “therese”}. These are displayed inintegrated correction display 625.

FIG. 9 illustrates an alternative design where the virtual keyboardlayout 630 is the standard desktop QWERTY keyboard layout, and thecorrective display 635 is illustrated at the top of the keyboard. Acontinuous stroke 636 is also shown.

In alternative embodiment, not only complete words are recognized butalso substrings of words. For example the substring “co” can refer tothe word “copy”, the word “comment” and the word “columns”. FIG. 10illustrates an embodiment with a virtual keyboard 640 where uncompletedstroke 646 is recognized by a continuous stroke recognizer 205 to aranked preview set of close matching words 645 that have substringsrepresented.

In an alternative embodiment, command names are recognized by acontinuous stroke recognizer 205, previewed in a correction display andoutputted by an output unit. When the output unit decides a command namehas been inputted, for example “Undo”, the output unit sends therecognized command. For example, in one embodiment the output unit sendsthe command “Undo” by synthesizing key presses for pressing Ctrl+Z,which is used in the Microsoft Windows operating system to signal the“Undo” command.

In an alternative embodiment, command names are recognized by substringsof the command names as well as complete command names. FIG. 12Aillustrates an uncompleted continuous stroke 740 on a virtual keyboard700 that is recognized as the command “cut” because continuous stroke740 is a close match to the substring Command-C that is a substring ofthe command sequence Command-C-U-T for “cut”. In this example, thecontinuous stroke 740 intersects both a command key 730 and the letter“c”. The desired command, “Copy”, is displayed in the left display 720,but because the input sequence “Command-C” also matches “Cut”, “Cut” ispreviewed in the primary display 710 (because, in this example, it isthe shorter 6word). To select the desired command, “Copy”, the usercould simply quickly move his or her pen or finger toward “Copy” on theleft display 720 or continue the stroke at normal speed toward theletter “o” on the keyboard 700.

FIG. 12B illustrates the same virtual keyboard 700 with an uncompletedcontinued continuous stroke 780 from FIG. 12A where continuous stroke780 is a close match the command “copy” because continuous stroke 780 isa close match to the substring of the command sequence Command-C-0 for“copy”, touching upon the command key 730, and then the “c” and “o”keys. Thus, the primary display 760 and alternative display 770 changeto reflect further movement in the continuous stroke 780.

While there has been described what is believed to be the preferredembodiment of the present invention, those skilled in the art willrecognize that other and further changes and modifications may be madethereto without departing from the spirit of the invention. Therefore,the invention is not limited to the specific details and representativeembodiments shown and described herein. Accordingly, variousmodifications may be made without departing from the spirit or scope ofthe general inventive concept as defined and differentiated by thefollowing claims.

I/we claim:
 1. A computing system for receiving input with a continuousstroke, the computing system comprising: one or more processors; amemory; a display; and an interface configured to: display a virtualkeyboard; and receive a first part of the continuous stroke which hasbeen input into the virtual keyboard, wherein the first part of thecontinuous stroke corresponds to locations, on the virtual keyboard, ofmultiple letters of the beginning of a word in sequence, withoutindicating a last letter of the word; wherein the memory storesinstructions that, when executed by the computing system, cause thecomputing system to perform operations that: determine, while the firstpart of the continuous stroke is being received through the interface,and before the last letter corresponding to the first part of thecontinuous stroke has been indicated, a matching set of word patternsthat at least partially match the first part of the continuous stroke,obtain, corresponding to the matching set of word patterns,representations of multiple words which correspond to the received firstpart of the continuous stroke; and wherein the display is configured to,in response to a trigger event, display at least one of therepresentations of the multiple words before the continuous stroke hasbeen completed, wherein the trigger event is based on a comparisonbetween speed information concerning a current rate of movement ofentering the continuous stroke and a threshold speed value.
 2. Thesystem of claim 1, wherein the comparison between speed informationconcerning the current rate of movement of entering the continuousstroke and the threshold speed value results in the trigger event whenthe comparison determines the current rate of movement of entering thecontinuous stroke is below the threshold speed value.
 3. A method forreceiving input with a continuous stroke, the method comprising:receiving a first part of the continuous stroke which has been inputinto a virtual keyboard, wherein the first part of the continuous strokecorresponds to areas, on the virtual keyboard, of multiple characters ofthe beginning of a set of characters, without indicating a last letterof the set of characters; while the first part of the continuous strokeis being received and before the last letter corresponding to the firstpart of the continuous stroke has been indicated, determining a matchingset of patterns that at least partially match the first part of thecontinuous stroke, obtaining, from the matching set of word patterns,representations of multiple sets of characters that correspond to thereceived first part of the continuous stroke; and displaying, inresponse to a comparison event, at least one of the representations ofthe multiple words before the continuous stroke has been completed,wherein the comparison event comprises a comparison between speedinformation concerning a current rate of movement of entering thecontinuous stroke and a threshold speed value.
 4. The method of claim 3further comprising: receiving a selection of a selected one of themultiple sets of characters by receiving an extension of the continuousstroke to intersect with a displayed representation of the selected oneof the multiple sets of characters.
 5. The method of claim 3 furthercomprising: receiving a selection of a selected one of the multiple setsof characters by determining, prior to completing the continuous stroke,that an input implement has lifted from the virtual keyboard.
 6. Themethod of claim 3 further comprising: receiving a selection of aselected one of the multiple sets of characters by determining that aninput implement has moved, at or above a predetermined threshold speed,outside the virtual keyboard.
 7. The method of claim 3, whereindisplaying at least one of the representations of the multiple sets ofcharacters comprises displaying a best matching set of characters, ofthe multiple sets of characters, with a distinct graphical attribute. 8.The method of claim 3, wherein the method is performed by one of: acomputer, a PDA, a cellular phone, a smartphone, a tablet PC, a mobilecommunications device, or a projection device having a touch-screen. 9.The method of claim 3, wherein the comparison between speed informationconcerning the current rate of movement of entering the continuousstroke and the threshold speed value results in the comparison eventwhen the comparison determines the current rate of movement of enteringthe continuous stroke is below the threshold speed value.
 10. The methodof claim 3, wherein the comparison between speed information concerningthe current rate of movement of entering the continuous stroke and thethreshold speed value results in the comparison event when thecomparison determines the current rate of movement of entering thecontinuous stroke is at or above the threshold speed value.
 11. Themethod of claim 3: wherein the continuous stroke is associated with anactive application; and wherein the method further comprises: receivinga selection of one of the multiple sets of characters; and deleting,from the associated active application, the selected one of the multiplesets of characters.
 12. The method of claim 3: wherein the continuousstroke is associated with an active application; and wherein the methodfurther comprises: receiving a selection of one of the multiple sets ofcharacters; and outputting, to the associated active application, theselected one of the multiple sets of characters.
 13. The method of claim3, wherein the continuous stroke is associated with an activeapplication of a computing system; wherein the method further comprises:receiving a selection of one of the multiple sets of characters; andoutputting, to the associated active application, the selected one ofthe multiple sets of characters; and wherein the associated activeapplication is an operating system of the computing system.
 14. Themethod of claim 3, wherein the continuous stroke is associated with anactive application; wherein the method further comprises: receiving aselection of one of the multiple sets of characters; and outputting, tothe associated active application, the selected one of the multiple setsof characters; and wherein the associated active application is a wordprocessor, an e-mail client, or an instant messaging client.
 15. Acomputer-readable storage medium storing instructions that, whenexecuted by a computing system, cause the computing system to performoperations for receiving input through a continuous stroke, theoperations comprising: receiving a first part of the continuous strokewhich has been input into a virtual keyboard, wherein the first part ofthe continuous stroke corresponds to areas, on the virtual keyboard, ofmultiple characters of the beginning of a set of characters, withoutindicating a last letter of the set of characters; while the first partof the continuous stroke is being received and before the last lettercorresponding to the first part of the continuous stroke has beenindicated, determining a matching set of patterns that at leastpartially match the first part of the continuous stroke, obtaining, fromthe matching set of word patterns, representations of multiple sets ofcharacters which correspond to the received first part of the continuousstroke; and displaying, in response to a comparison event, at least oneof the representations of the multiple words before the continuousstroke has been completed, wherein the comparison event comprises acomparison between velocity information concerning a current rate ofmovement of entering the continuous stroke and a threshold velocityvalue.
 16. The computer-readable storage medium of claim 15, wherein theoperations further comprise: receiving a selection of a selected one ofthe multiple sets of characters by receiving an extension of thecontinuous stroke to intersect with a displayed representation of theselected one of the multiple sets of characters.
 17. Thecomputer-readable storage medium of claim 15, wherein the operationsfurther comprise: receiving a selection of a selected one of themultiple sets of characters by determining, prior to completing thecontinuous stroke, that an input has lifted from the virtual keyboard.18. The computer-readable storage medium of claim 15, wherein thecomparison between velocity information concerning the current rate ofmovement of entering the continuous stroke and the threshold velocityvalue results in the comparison event when the comparison determines thecurrent rate of movement of entering the continuous stroke is below thethreshold velocity value.
 19. The computer-readable storage medium ofclaim 15, wherein the comparison between velocity information concerningthe current rate of movement of entering the continuous stroke and thethreshold velocity value results in the comparison event when thecomparison determines the current rate of movement of entering thecontinuous stroke is at or above the threshold velocity value.
 20. Thecomputer-readable storage medium of claim 15: wherein the continuousstroke is associated with an active application; and wherein theoperations further comprise: receiving a selection of one of themultiple sets of characters; and outputting, to the associated activeapplication, the selected one of the multiple sets of characters.